The Internet is a global public network system of interconnected computer networks. It consists of millions of private, public, business and government networks that are all interconnected through a series of electronic, wireless and/or optical connections that are typically maintained by governmental entities and internet service providers (ISPs).
Content provider is a term used to describe an entity that distributes typically informational, educational, and/or entertainment content over the networks. Typically the content is accessed via websites using hypertext transport protocols (HTTP) and is mixture of internet content applications such as text files, image files, audio/video files, or application programs. Users navigate the Internet with a browser, which is a program designed to transfer information on the Internet. When a user types in a website's universal resource locator (URL) into the browser the information is transmitted to the ISP. The ISP routes that information to the corresponding content provider's website, which then transmits data back to the user including the content.
Every year the amount of content being added to the Internet continues to grow and the amount of people with high-speed internet access also grows. Likewise, many popular content provider websites like google.com, yahoo.com, hulu.com or youtube.com are visited by new users every day. Because of the ever-increasing internet traffic, there was a need to reliably, economically, and efficiently provide ever-richer content to an ever-growing number of users. To accomplish that goal, many content providers use content delivery networks (CDNs). CDNs are networks of server computers that store copies of the content and are typically distributed throughout the country and the world. In short, CDNs are storage centers that allow content providers to store multiple copies of their content in networks around the country, instead of in one single location. Some companies own and operate their own CDNs, while others will use a service from a third party.
There are benefits to storing and distributing content using a CDN. All the network traffic is not being routed to a single a location (URL) creating a bottleneck at the content provider. Rather, users are directed to one of the many CDNs to reduce strain on the network. Generally, the use of CDN also increases performance and decreases download times because the users accessing the content can be directed to CDNs that are local to the users, often avoiding the need to transmit the data over long distances. This reduces latency and decreases traffic on major internet backbones. Additionally, there is a built in redundancy. If a CDN network is unavailable because of a power outage or even lost completely due to a disaster, the other CDNs are still able to provide content to users.
By way of example, a user will navigate to a content provider's website and select web content to access. The content could be any internet content application such as a video, audio, or image file or application program. The website will then direct the user to URL or internet protocol (IP) address hosted by a CDN, where the content is stored, and the user will receive the content from the CDN. Generally, the entire process of delivering the content to user is designed to be a seamless and transparent operation providing an experience that leaves the user unaware that the content was accessed from a CDN, and not the content provider's website.
At the enterprise, ISP, and public network scale, network management systems are used to monitor networks. These systems can exist as stand-alone, dedicated systems or be embedded in network communications devices such as routers and switches. One specific example is NetFlow technology offered by Cisco Systems. Other tools include special-purpose systems, such as firewalls and other network security devices, which are typically used to manage the communications at boundaries between the networks.
One source of information for monitoring networks is flow information. This is defined as “a unidirectional sequence of packets with some common properties that pass through a network device.” Internet Engineering Task Force, RFC 3954. Flow records are often generated by the network devices. These are often digested information concerning individual network flows or groups of network flows sharing some common characteristic(s). The flow records often include, for example, internet protocol (IP) addresses, packet and byte counts, timestamps, Type of Service (ToS), application ports, input and output interfaces, to list a few examples. This information is available from Netflow technology, for example. Generally, computer network devices that generate flow records include, for example, routers, switches, firewalls, and hubs. In other examples, packet scanners/analyzers (e.g. Arbor Networks PEAKFLOW® threat management system (TMS)) are used. Flows may be collected and exported for analysis. Flow analysis is a central component of large-scale network management and service systems, usually deployed by ISPs.
Network management systems allow the network administrators to apply policies. Policies are typically used to govern or dictate how entities are allowed to communicate over the network, generally called security policies. These policies can be applied to entities individually, by setting operating parameters of devices separately. Policy-based management systems have simplified configuration of devices by allowing administrators to define a policy and apply this policy across groups of network entities, generally.
A policy is a collection of rules. A rule, for example, can be defined to govern what traffic a particular firewall ignores or prevents a given address or device from accessing a particular service or network resource. The rules can be applied by routers that decide whether to forward packets from or to a particular address.
Network policies are often defined and applied based on flow information. Moreover, many products are available that attempt to correlate flow information with other data sources to provide value-added analysis. These types of analysis tools are now a central component of administering large communication networks. Such analysis facilitates the creation of higher level policies that facilitate the management of the network.